Skip to content

Conversation

naveenpaul1
Copy link
Contributor

@naveenpaul1 naveenpaul1 commented Jul 29, 2025

Describe the Problem

Explain the Changes

  1. Upgrade AWS SDK to v3 - NamespaceS3
  2. Remove AWS SDK V2 cloud util changes

Issues: Fixed #xxx / Gap #xxx

  1. Remove deprecated AWS SDKv3 for NamespaceS3.

Testing Instructions:

  1. Run namespaces3 flows
  • Doc added/updated
  • Tests added

Reference: https://mattsumme.rs/2025/aws-sdk-v3-put-object-stream-regression/
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/migrating/notable-changes/
https://docs.aws.amazon.com/AWSJavaScriptSDK/v3/latest/Package/-aws-sdk-client-s3/Interface/PutObjectOutput/

Summary by CodeRabbit

  • Refactor

    • Migrated S3 integration to AWS SDK v3-style: improved streaming, multipart uploads, request handling, richer object metadata and operation-date capture, and stricter bucket requirement.
  • Chores

    • Removed legacy v2 credential/client paths and deprecated public helpers; consolidated on STS v3 and v3-style S3 client configuration.
  • Tests

    • Updated tests to exercise SDK v3 flows and removed obsolete v2-specific coverage.

Copy link

coderabbitai bot commented Jul 29, 2025

Walkthrough

Removed two S3 helper exports and migrated S3/STS usage from AWS SDK v2 to v3 across SDK and utility code; NamespaceS3 now requires a top-level bucket argument, replaces .promise() flows with async/await v3 calls, and test coverage for STS v2 was removed while v3 tests remain. (≤50 words)

Changes

Cohort / File(s) Summary of changes
S3 helper removal
src/endpoint/s3/s3_utils.js
Deleted two exported helpers: get_http_response_date and get_http_response_from_resp.
NamespaceS3 (v3 migration)
src/sdk/namespace_s3.js
Migrated to AWS SDK v3: constructor now requires top-level bucket; uses v3 client calls (await) for list/get/put/copy/multipart/tagging/ACL; updated types, error handling, multipart flows, date/etag extraction and logging; removed .promise() patterns.
ObjectSDK S3 init (v3)
src/sdk/object_sdk.js
Build v3-style s3_params (nested credentials, region, forcePathStyle, requestHandler via noobaa_s3_client, requestChecksumCalculation); pass bucket top-level into NamespaceS3; removed v2 httpOptions/agent patterns.
Cloud utils STS v3 adoption
src/util/cloud_utils.js
Removed legacy v2 STS paths/exports (createSTSS3Client, generate_aws_sts_creds); added/retained v3 flow (generate_aws_sdkv3_sts_creds, createSTSS3SDKv3Client) returning plain creds and constructing v3 S3 clients with NodeHttpHandler/requestHandler and default region.
Tests: STS v2 removal
src/test/unit_tests/util_functions_tests/test_cloud_utils.js
Removed AWS SDK v2 STS tests and v2-specific constants; retained SDK v3 STS tests.
Tests: integration credentials sourcing
src/test/integration_tests/api/s3/test_s3_ops.js
Minor whitespace cleanup in mocha.before; functional behavior unchanged.

Sequence Diagram(s)

sequenceDiagram
  participant App as ObjectSDK
  participant NS as NamespaceS3
  participant S3 as S3Client (v3)
  participant RH as RequestHandler

  Note over App: Build v3 s3_params + requestHandler
  App->>RH: get_requestHandler_with_suitable_agent(endpoint)
  App->>NS: new NamespaceS3({ namespace_resource_id, s3_params, bucket, stats })
  NS->>S3: Init S3 client with s3_params + requestHandler

  rect rgba(223,242,225,0.6)
    Note over App,NS: Read object stream (v3 flow)
    App->>NS: read_object_stream(key, range?)
    NS->>S3: getObject / headObject (await)
    S3-->>NS: Response (metadata + Body stream / error)
    NS->>App: Return bridged Readable stream, update stats
  end
Loading
sequenceDiagram
  participant Caller as Cloud Utils Consumer
  participant STS as STSClient (v3)
  participant CloudUtils as cloud_utils

  Caller->>CloudUtils: generate_aws_sdkv3_sts_creds(params)
  CloudUtils->>STS: ExecuteRoleWithWebIdentityCommand (await)
  STS-->>CloudUtils: { AccessKeyId, SecretAccessKey, SessionToken }
  CloudUtils->>Caller: plain credentials object
  Caller->>CloudUtils: createSTSS3SDKv3Client(credentials, endpoint)
  CloudUtils->>CloudUtils: build S3 client with NodeHttpHandler/requestHandler
Loading

Estimated code review effort

🎯 4 (Complex) | ⏱️ ~45 minutes

Possibly related PRs

Suggested reviewers

  • dannyzaken
  • liranmauda
  • tangledbytes

Tip

🔌 Remote MCP (Model Context Protocol) integration is now available!

Pro plan users can now connect to remote MCP servers from the Integrations page. Connect with popular remote MCPs such as Notion and Linear to add more context to your reviews and chats.

✨ Finishing Touches
  • 📝 Generate Docstrings
🧪 Generate unit tests
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

CodeRabbit Commands (Invoked using PR/Issue comments)

Type @coderabbitai help to get the list of available commands.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

Status, Documentation and Community

  • Visit our Status Page to check the current availability of CodeRabbit.
  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

@naveenpaul1 naveenpaul1 marked this pull request as ready for review August 14, 2025 10:08
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 7

🔭 Outside diff range comments (1)
src/sdk/namespace_s3.js (1)

126-132: Correct pagination field: use NextUploadIdMarker.

The response’s next marker is NextUploadIdMarker. Returning UploadIdMarker will regress pagination.

Apply this diff:

         return {
             objects: _.map(res.Uploads, obj => this._get_s3_object_info(obj, params.bucket)),
             common_prefixes: _.map(res.CommonPrefixes, 'Prefix'),
             is_truncated: res.IsTruncated,
             next_marker: res.NextKeyMarker,
-            next_upload_id_marker: res.UploadIdMarker,
+            next_upload_id_marker: res.NextUploadIdMarker,
         };
🧹 Nitpick comments (2)
src/endpoint/s3/s3_utils.js (1)

671-673: Align error message with v3 shape and consider normalizing the return type.

  • The message references "response header" while v3 exposes LastModified as a top-level field. Nit but confusing.
  • Optional: consider returning a numeric timestamp for consistency with other paths that use getTime().

Apply this diff for the message:

-    if (!r.LastModified) throw new Error("date not found in response header");
+    if (!r.LastModified) throw new Error("LastModified not found on response");
src/sdk/namespace_s3.js (1)

862-896: Outdated JSDoc types for SSE fields (v2 types).

These reference AWS SDK v2 types. Consider updating to v3 inputs for clarity:

  • ServerSideEncryption -> import("@aws-sdk/client-s3").ServerSideEncryption
  • SSEKMSKeyId -> string, etc.

I can open a quick PR to refresh these annotations if you want.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these settings in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 704e34d and 9704f13.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (1 hunks)
  • src/sdk/namespace_s3.js (32 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (3)
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/util/cloud_utils.js
🧰 Additional context used
🧬 Code Graph Analysis (1)
src/sdk/namespace_s3.js (3)
src/sdk/object_sdk.js (3)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • _ (4-4)
src/util/cloud_utils.js (5)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • _ (11-11)
src/endpoint/s3/s3_utils.js (5)
  • err (421-421)
  • err (445-445)
  • err (538-538)
  • err (547-547)
  • _ (4-4)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: run-jest-unit-tests
🔇 Additional comments (1)
src/sdk/object_sdk.js (1)

28-29: LGTM: importing the v3-aware request handler.

This aligns construction with SDK v3 and allows per-endpoint agent selection. Good move.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (1)
src/sdk/object_sdk.js (1)

463-474: Restore signatureVersion selection to preserve SigV2 endpoints and SDK v2 fallback.

Without signatureVersion, we effectively force SDKv3/SigV4-only. Deployments/endpoints that still require SigV2 (e.g., resources configured with AWS_V2) will break because noobaa_s3_client.should_use_sdk_v2 won’t have a signal to choose SDK v2.

Apply this change:

                 s3_params: {
                     endpoint: r.endpoint,
                     aws_sts_arn: r.aws_sts_arn,
+                    signatureVersion: cloud_utils.get_s3_endpoint_signature_ver(r.endpoint, r.auth_method),
                     credentials: {
                         accessKeyId: r.access_key.unwrap(),
                         secretAccessKey: r.secret_key.unwrap(),
                     },
                     region: config.DEFAULT_REGION, // SDKv3 needs region
                     forcePathStyle: true,
                     requestHandler: noobaa_s3_client.get_requestHandler_with_suitable_agent(r.endpoint),
                     access_mode: r.access_mode,
                 },
🧹 Nitpick comments (2)
src/sdk/object_sdk.js (2)

28-28: Nit: Consider lazy-requiring noobaa_s3_client to avoid top-level cost/cycles.

object_sdk is a hot path. Requiring the S3 client wrapper at module load time can increase startup cost and risks a circular require with NamespaceS3. Suggest moving this require into the S3 branch of _setup_single_namespace (just before creating NamespaceS3) to defer it until needed.


470-473: Region default and request handler: add safe fallback and ensure handler reuse.

  • Region: Some S3-compatible endpoints expect us-east-1 (or ignore region). To reduce signature-mismatch risk when config is unset, prefer a safe fallback.
  • Handler: Ensure get_requestHandler_with_suitable_agent caches/returns a shared NodeHttpHandler per endpoint (keep-alive, maxSockets) and respects HTTP(S)_PROXY env vars to avoid socket churn and honor proxies.

Suggested tweak for region:

-                    region: config.DEFAULT_REGION, // SDKv3 needs region
+                    region: r.region || config.DEFAULT_REGION || 'us-east-1', // SDKv3 needs region; fall back safely
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9704f13 and f5ec83c.

📒 Files selected for processing (2)
  • src/sdk/namespace_s3.js (32 hunks)
  • src/sdk/object_sdk.js (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/sdk/namespace_s3.js
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-jest-unit-tests
  • GitHub Check: run-package-lock-validation
🔇 Additional comments (2)
src/sdk/object_sdk.js (2)

466-469: Credentials nesting matches SDK v3 config expectations.

Moving access keys under credentials is correct for S3Client v3.


475-475: Passing bucket as a top-level arg is consistent with the new NamespaceS3 signature.

This aligns with the v3 refactor and simplifies s3_params.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (3)
src/endpoint/s3/s3_utils.js (1)

676-678: Broaden HTTP status acceptance to any 2xx (SDK v3 returns 206 for ranged GET/HEAD).

Only allowing 200 will break ranged operations and some HEADs under v3.

Apply this diff:

-function get_http_response_from_resp(res) {
-    if (res.$metadata.httpStatusCode !== 200) throw new Error("Response return with error");
-    return res;
-}
+function get_http_response_from_resp(res) {
+    const code = res?.$metadata?.httpStatusCode;
+    if (!code || Math.floor(code / 100) !== 2) {
+        throw new Error(`Response returned with error (status ${code})`);
+    }
+    return res;
+}
src/sdk/namespace_s3.js (1)

480-489: Don’t stringify PartNumberMarker; pass number or undefined.

S3 expects a number. Stringifying is unnecessary and can be surprising.

-        const req = {
+        const req = {
             Bucket: this.bucket,
             Key: params.key,
             UploadId: params.obj_id,
             MaxParts: params.max,
-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,
         };
src/sdk/object_sdk.js (1)

466-475: Restore SigV2 support in ObjectSDK S3 client parameters

You’ve removed the signatureVersion setting from src/sdk/object_sdk.js, which means all S3 clients will default to SigV4—even IBM_COS endpoints that require SigV2 will break. Re-add the signature selector based on r.auth_method:

• File: src/sdk/object_sdk.js
Location: inside the s3_params object for NamespaceS3 (around line 471)

Suggested patch:

                     forcePathStyle: true,
                     requestHandler: noobaa_s3_client.get_requestHandler_with_suitable_agent(r.endpoint),
                     requestChecksumCalculation: 'WHEN_REQUIRED',
+                    signatureVersion: cloud_utils.get_s3_endpoint_signature_ver(r.endpoint, r.auth_method),
                     access_mode: r.access_mode,

This restores the V2/V4 switch via cloud_utils.get_s3_endpoint_signature_ver. If SigV2 is truly deprecated across all pools, remove IBM_COS from config.DEFAULT_S3_AUTH_METHOD and update docs/tests to reflect a SigV4-only policy.

🧹 Nitpick comments (6)
src/endpoint/s3/s3_utils.js (1)

671-673: Clarify error and keep semantics consistent (LastModified).

The function now relies on LastModified, so the error message should reflect that. Callers currently expect a Date (not a string), which is fine.

-    if (!r.LastModified) throw new Error("date not found in response header");
+    if (!r.LastModified) throw new Error("LastModified not found in response");
     return r.LastModified;
src/sdk/namespace_s3.js (5)

32-43: Use v3 credentials shape when setting access key.

With v3 you’re passing credentials under s3_params.credentials, so this.access_key will be undefined and server-side copy checks will always fail.

-        this.access_key = s3_params.accessKeyId;
+        this.access_key = s3_params?.credentials?.accessKeyId || s3_params.accessKeyId;

249-256: Tighten request typing: only GetObjectRequest is used.

JSDoc union includes CopyObjectRequest which isn’t relevant to read path.

-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & import("@aws-sdk/client-s3").GetObjectRequest | import("@aws-sdk/client-s3").CopyObjectRequest} */
+        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & import("@aws-sdk/client-s3").GetObjectRequest} */
         const request = {

265-266: Superfluous status guard (v3 throws on non-2xx).

The extra check is redundant and can hide null-safety issues; removing it simplifies the flow.

-            // In v3, non-2xx typically throws; keep this guard harmless.
-            if (obj_out.$metadata.httpStatusCode >= 300) throw new Error(`S3 getObject failed with status ${obj_out.$metadata.httpStatusCode}`);
+            // v3 throws on non-2xx; no additional guard needed

361-369: Head the created version and normalize last_modified_time.

Use the returned VersionId (if present) to head the exact created version. Also return epoch millis for consistency with other paths.

-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest} */
-        const request = {
-            Bucket: this.bucket,
-            Key: params.key,
-            VersionId: params.version_id,
-        };
-        const res_head = await this.s3.headObject(request);
-        const last_modified_time = await s3_utils.get_http_response_date(res_head);
+        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest} */
+        const request = {
+            Bucket: this.bucket,
+            Key: params.key,
+            VersionId: res.VersionId || params.version_id,
+        };
+        const res_head = await this.s3.headObject(request);
+        const last_modified_time = s3_utils.get_http_response_date(res_head)?.getTime?.();

852-866: Broaden error translation for v3 error shapes (name/$metadata).

err.code is often undefined in v3. Use err.name and HTTP status to preserve mappings.

-    _translate_error_code(params, err) {
-        if (err.code === 'NoSuchKey') err.rpc_code = 'NO_SUCH_OBJECT';
-        else if (err.code === 'NotFound') err.rpc_code = 'NO_SUCH_OBJECT';
-        else if (err.code === 'InvalidRange') err.rpc_code = 'INVALID_RANGE';
-        else if (params.md_conditions) {
-            const md_conditions = params.md_conditions;
-            if (err.code === 'PreconditionFailed') {
-                if (md_conditions.if_match_etag) err.rpc_code = 'IF_MATCH_ETAG';
-                else if (md_conditions.if_unmodified_since) err.rpc_code = 'IF_UNMODIFIED_SINCE';
-            } else if (err.code === 'NotModified') {
-                if (md_conditions.if_modified_since) err.rpc_code = 'IF_MODIFIED_SINCE';
-                else if (md_conditions.if_none_match_etag) err.rpc_code = 'IF_NONE_MATCH_ETAG';
-            }
-        }
-    }
+    _translate_error_code(params, err) {
+        const code = err?.code || err?.name;
+        const http = err?.$metadata?.httpStatusCode;
+        if (code === 'NoSuchKey' || code === 'NotFound' || http === 404) {
+            err.rpc_code = 'NO_SUCH_OBJECT';
+        } else if (code === 'InvalidRange' || http === 416) {
+            err.rpc_code = 'INVALID_RANGE';
+        } else if (params.md_conditions) {
+            const md_conditions = params.md_conditions;
+            if (code === 'PreconditionFailed' || http === 412) {
+                if (md_conditions.if_match_etag) err.rpc_code = 'IF_MATCH_ETAG';
+                else if (md_conditions.if_unmodified_since) err.rpc_code = 'IF_UNMODIFIED_SINCE';
+            } else if (code === 'NotModified' || http === 304) {
+                if (md_conditions.if_modified_since) err.rpc_code = 'IF_MODIFIED_SINCE';
+                else if (md_conditions.if_none_match_etag) err.rpc_code = 'IF_NONE_MATCH_ETAG';
+            }
+        }
+    }
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between f5ec83c and 4b2e592.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (1 hunks)
  • src/sdk/namespace_s3.js (32 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (3)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/util/cloud_utils.js
  • src/test/integration_tests/api/s3/test_s3_ops.js
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/sdk/object_sdk.js (5)
src/sdk/namespace_s3.js (2)
  • noobaa_s3_client (14-14)
  • config (8-8)
src/util/cloud_utils.js (1)
  • noobaa_s3_client (16-16)
src/server/system_services/account_server.js (1)
  • noobaa_s3_client (26-26)
src/server/bg_services/namespace_monitor.js (2)
  • noobaa_s3_client (15-15)
  • config (13-13)
src/sdk/bucketspace_s3.js (1)
  • noobaa_s3_client (6-6)
src/sdk/namespace_s3.js (2)
src/sdk/object_sdk.js (3)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • _ (4-4)
src/util/cloud_utils.js (5)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • _ (11-11)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-package-lock-validation
  • GitHub Check: run-jest-unit-tests
🔇 Additional comments (3)
src/sdk/namespace_s3.js (2)

210-222: LGTM: v3-compatible InvalidRange handling.

Catching by err.name and checking $metadata.httpStatusCode === 416 covers v3 error shapes and keeps the fallback sane.


249-286: LGTM: streaming fix returns live Node readable and updates stats via tap.

This avoids pre-consuming the stream and uses v3’s Body as a Readable directly. Error propagation and rethrow are correct.

src/sdk/object_sdk.js (1)

466-475: LGTM: v3 client config wiring (credentials, region, forcePathStyle, requestHandler).

Solid move to v3 semantics; dynamic agent via requestHandler is appropriate.

Also applies to: 476-478

@naveenpaul1 naveenpaul1 changed the title SDK | Upgrade AWS SDK to v3 - Namespace_S3 SDK | Upgrade AWS SDK to v3 - NamespaceS3 Aug 19, 2025
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (2)
src/endpoint/s3/s3_utils.js (1)

670-672: Accept any 2xx status (v3 returns 206 for ranged responses) and improve error message.

Only accepting 200 will incorrectly throw on valid 206 responses (e.g., ranged GET/HEAD). Also tweak the message grammar and add null-safety.

Apply this diff:

-function get_http_response_date(res) {
-    if (res.$metadata.httpStatusCode !== 200) throw new Error("Response return with error");
-    if (!res.LastModified) throw new Error("Date not found in response header");
-    return res.LastModified;
-}
+function get_http_response_date(res) {
+    const code = res?.$metadata?.httpStatusCode;
+    if (!code || Math.floor(code / 100) !== 2) {
+        throw new Error(`Response returned with error (status ${code})`);
+    }
+    if (!res?.LastModified) throw new Error("Date not found in response header");
+    return res.LastModified;
+}
src/sdk/namespace_s3.js (1)

480-489: Don’t stringify PartNumberMarker; allow number or undefined. Also note inconsistency with the summary.

  • API expects a number; .toString() is unnecessary and will throw when num_marker is undefined.
  • The AI summary says this was addressed, but the code still stringifies.
-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,
🧹 Nitpick comments (3)
src/sdk/namespace_s3.js (3)

24-29: JSDoc: make bucket required to match runtime validation.

Constructor throws if bucket is missing, but the type still marks it optional (bucket?). Align the doc to avoid confusion.

-     *      bucket?: string,
+     *      bucket: string,

249-257: Nit: narrow request typing and drop the redundant 2xx guard.

  • The request here is only for getObject; the CopyObjectRequest in the union isn’t used.
  • v3 throws on non-2xx by default—explicit httpStatusCode >= 300 check is unnecessary.
-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & import("@aws-sdk/client-s3").GetObjectRequest | import("@aws-sdk/client-s3").CopyObjectRequest} */
+        /** @type {import("@aws-sdk/client-s3").GetObjectRequest} */
         const request = {
             Bucket: this.bucket,
             Key: params.key,
             Range: params.end ? `bytes=${params.start}-${params.end - 1}` : undefined,
             PartNumber: params.part_number,
         };
@@
-            // In v3, non-2xx typically throws; keep this guard harmless.
-            if (obj_out.$metadata.httpStatusCode >= 300) throw new Error(`S3 getObject failed with status ${obj_out.$metadata.httpStatusCode}`);
+            // v3 throws on non-2xx; no explicit httpStatusCode guard needed.

Also applies to: 264-266, 279-281


360-369: Use returned VersionId for HEAD and normalize last_modified_time to a number.

  • If versioning is enabled, head the specific version you just wrote (res.VersionId) rather than params.version_id.
  • Normalize last_modified_time to a number (ms) for consistency with other code paths that use .getTime().
-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest} */
-        const request = {
-            Bucket: this.bucket,
-            Key: params.key,
-            VersionId: params.version_id,
-        };
-        const res_head = await this.s3.headObject(request);
-        const last_modified_time = await s3_utils.get_http_response_date(res_head);
+        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest} */
+        const request = {
+            Bucket: this.bucket,
+            Key: params.key,
+            VersionId: res.VersionId, // prefer the just-created version when available
+        };
+        const res_head = await this.s3.headObject(request);
+        const last_modified_time = s3_utils.get_http_response_date(res_head).getTime();
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 4b2e592 and a1dca53.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (1 hunks)
  • src/sdk/namespace_s3.js (33 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (3)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/util/cloud_utils.js
  • src/test/integration_tests/api/s3/test_s3_ops.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/sdk/object_sdk.js
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/endpoint/s3/s3_utils.js (3)
src/test/integration_tests/api/s3/test_s3_ops.js (9)
  • res (109-109)
  • res (113-113)
  • res (119-119)
  • res (174-174)
  • res (246-246)
  • res (280-280)
  • res (309-309)
  • res (337-337)
  • res (364-364)
src/server/system_services/bucket_server.js (4)
  • res (456-457)
  • res (1307-1307)
  • res (1437-1439)
  • res (1907-1910)
src/endpoint/s3/s3_bucket_policy_utils.js (3)
  • res (132-132)
  • res (141-141)
  • res (147-147)
src/sdk/namespace_s3.js (3)
src/sdk/object_sdk.js (4)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • dbg (9-9)
  • _ (4-4)
src/util/cloud_utils.js (6)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • dbg (3-3)
  • _ (11-11)
src/server/system_services/bucket_server.js (14)
  • noobaa_s3_client (41-41)
  • s3_params (2048-2058)
  • bucket (343-343)
  • bucket (357-357)
  • bucket (376-376)
  • bucket (394-394)
  • bucket (419-419)
  • bucket (431-431)
  • bucket (446-446)
  • bucket (482-482)
  • bucket (491-491)
  • req (1116-1116)
  • req (1149-1149)
  • req (2013-2013)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-package-lock-validation
  • GitHub Check: run-jest-unit-tests
🔇 Additional comments (4)
src/sdk/namespace_s3.js (4)

190-197: Good v3-compatible InvalidRange handling.

Catching by err.name === 'InvalidRange' || err?.$metadata?.httpStatusCode === 416 and falling back to headObject for zero-length objects is correct. Nice.

Also applies to: 209-221


434-437: ETag extraction for multipart copy/upload looks correct.

Using res.CopyPartResult.ETag for copy and res.ETag for upload is the right mapping for v3.

Also applies to: 471-473


764-775: Correct API usage for GetObjectAttributes (v3).

Typing and request composition look good, and attributes are passed as expected.


817-818: Nice addition of ObjectSize as a fallback for size.

This aligns with v3’s GetObjectAttributes response shape.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

♻️ Duplicate comments (2)
src/sdk/namespace_s3.js (2)

35-35: Access key extraction is incorrect for SDK v3 credentials

The current approach of extracting accessKeyId directly from s3_params will be undefined in SDK v3, as credentials are typically provided under s3_params.credentials. This could lead to incorrect server-side copy decisions when both namespaces incorrectly compare as undefined.

Apply this diff to properly extract the access key:

-        this.access_key = s3_params.accessKeyId;
+        this.access_key = s3_params.credentials?.accessKeyId || s3_params.accessKeyId;

Also verify this works with async credential providers if they're used in your setup.


494-494: Handle undefined num_marker correctly

When params.num_marker is undefined, calling .toString() will throw an error. The SDK v3 expects a string for PartNumberMarker, but it's better to omit the field when undefined.

Apply this diff to handle undefined values:

-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker ? params.num_marker.toString() : undefined,
🧹 Nitpick comments (1)
src/sdk/namespace_s3.js (1)

365-366: Fix typo in error message

There's a typo in the error log message.

-                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and 
-                    key : ${params.key}, with erro : `, err);
+                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and 
+                    key : ${params.key}, with error : `, err);
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between a1dca53 and c9abf9c.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (0 hunks)
  • src/sdk/namespace_s3.js (34 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (4)
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/endpoint/s3/s3_utils.js
  • src/util/cloud_utils.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/sdk/object_sdk.js
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: run-package-lock-validation
  • GitHub Check: run-jest-unit-tests
  • GitHub Check: Build Noobaa Image
🔇 Additional comments (5)
src/sdk/namespace_s3.js (5)

249-286: LGTM! Stream handling properly migrated to v3

The migration correctly:

  • Uses the native Node.js Readable stream from obj_out.Body
  • Returns a live piped stream for the caller to consume
  • Properly handles errors and re-throws them
  • Updates read statistics via the tap stream

213-219: LGTM! Proper v3 error handling for InvalidRange

The error handling correctly checks both the error name and HTTP status code (416) to detect InvalidRange errors in SDK v3, providing proper fallback to headObject for zero-length objects.


443-444: LGTM! Correct ETag extraction for uploadPartCopy

The code correctly extracts the ETag from res.CopyPartResult.ETag for copy operations, which is the proper location in SDK v3 responses.


376-376: LGTM! Proper ETag extraction handling both copy and regular uploads

The code correctly handles ETag extraction for both regular uploads (res.ETag) and copy operations (res.CopyObjectResult?.ETag).


349-377: Outdated suggestion – no HeadObject call to avoid
The current implementation doesn’t invoke headObject at all. It cleverly captures the response’s Date header via middleware on the single putObject call, so there isn’t a doubled API request. You can safely ignore the recommendation to avoid a non-existent headObject call.

Likely an incorrect or invalid review comment.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (4)
src/sdk/namespace_s3.js (4)

298-377: Middleware leak and concurrency hazard when capturing Date on PutObject.

  • Adding a named middleware on the shared client and removing it only on success leaks the middleware on failures. Over time you’ll accumulate duplicate middlewares.
  • Concurrent uploads share this.s3; using a single static name (“saveDateMiddleware”) risks collisions between requests.

Use a unique name per invocation and remove in finally. Also guard header access and fix a minor typo in the error log. Optional: bridge source_stream errors to the counting stream as well.

-            let count = 1;
+            let count = 1;
             const count_stream = stream_utils.get_tap_stream(data => {
                 this.stats?.update_namespace_write_stats({
                     namespace_resource_id: this.namespace_resource_id,
                     bucket_name: params.bucket,
                     size: data.length,
                     count
                 });
                 // clear count for next updates
                 count = 0;
             });
-
+            // propagate upstream errors to the body being sent
+            params.source_stream.on?.('error', err => count_stream.emit('error', err));
             /** @type {import("@aws-sdk/client-s3").PutObjectRequest} */
             const request = {
                 Bucket: this.bucket,
                 Key: params.key,
                 Body: params.source_stream.pipe(count_stream),
                 ContentLength: params.size,
                 ContentType: params.content_type,
                 ContentMD5: params.md5_b64,
                 Metadata: params.xattr,
                 Tagging,
             };
             this._assign_encryption_to_request(params, request);
-             try {
-                // Add a middleware to log response headers
-                this.s3.middlewareStack.add(
-                    (next, context) => async args => {
-                        const result = await next(args);
-                        date = new Date(result.response.headers.date);
-                        return result;
-                    }, {
-                        step: "deserialize", // you could also use "deserialize"
-                        name: "saveDateMiddleware",
-                        tags: ["DATE_MIDDLEWARE"],
-                    }
-                );
-                res = await this.s3.putObject(request);
-                this.s3.middlewareStack.remove("saveDateMiddleware");
-            } catch (err) {
-                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and 
-                    key : ${params.key}, with erro : `, err);
+            const middlewareName = `saveDateMiddleware:${Date.now()}:${Math.random().toString(36).slice(2)}`;
+            try {
+                // Add a per-call middleware to capture the Date response header
+                this.s3.middlewareStack.add(
+                    (next, _context) => async args => {
+                        const result = await next(args);
+                        const hdrs = result?.response?.headers || {};
+                        const hdrDate = hdrs.date || hdrs.Date;
+                        if (hdrDate) {
+                            date = new Date(hdrDate);
+                        }
+                        return result;
+                    },
+                    { step: "deserialize", name: middlewareName, tags: ["DATE_MIDDLEWARE"] }
+                );
+                res = await this.s3.putObject(request);
+            } catch (err) {
+                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and key: ${params.key}, error:`, err);
                 object_sdk.rpc_client.pool.update_issues_report({
                     namespace_resource_id: this.namespace_resource_id,
                     error_code: String(err.code),
                     time: Date.now(),
                 });
                 throw err;
+            } finally {
+                try { this.s3.middlewareStack.remove(middlewareName); } catch (_) {}
             }

426-471: Bridge errors from source_stream in multipart upload body.

Same stream error propagation concern applies here. If params.source_stream errors, the Body (count_stream) won’t emit it to callers/S3 client.

             let count = 1;
             const count_stream = stream_utils.get_tap_stream(data => {
                 this.stats?.update_namespace_write_stats({
                     namespace_resource_id: this.namespace_resource_id,
                     size: data.length,
                     count
                 });
                 // clear count for next updates
                 count = 0;
             });
 
             /** @type {import("@aws-sdk/client-s3").UploadPartRequest} */
             const request = {
                 Bucket: this.bucket,
                 Key: params.key,
                 UploadId: params.obj_id,
                 PartNumber: params.num,
-                Body: params.source_stream.pipe(count_stream),
+                Body: params.source_stream.on?.('error', err => count_stream.emit('error', err)).pipe(count_stream),
                 ContentMD5: params.md5_b64,
                 ContentLength: params.size,
             };

860-873: Broaden v3 error translation; err.code is often unset.

Many v3 exceptions set name and httpStatusCode instead of code. Current mapping may miss common cases (404/416/412/304), affecting downstream rpc_code and behavior.

Here is a drop-in replacement for _translate_error_code (outside the changed range), preserving existing semantics while adding v3 checks:

_translate_error_code(params, err) {
    const http = err?.$metadata?.httpStatusCode;
    const code = err?.code || err?.name;

    const is404 = code === 'NoSuchKey' || code === 'NotFound' || http === 404;
    const is416 = code === 'InvalidRange' || http === 416;

    if (is404) {
        err.rpc_code = 'NO_SUCH_OBJECT';
        return;
    }
    if (is416) {
        err.rpc_code = 'INVALID_RANGE';
        return;
    }

    if (params.md_conditions) {
        const md_conditions = params.md_conditions;
        const is412 = code === 'PreconditionFailed' || http === 412;
        const is304 = code === 'NotModified' || http === 304;

        if (is412) {
            if (md_conditions.if_match_etag) err.rpc_code = 'IF_MATCH_ETAG';
            else if (md_conditions.if_unmodified_since) err.rpc_code = 'IF_UNMODIFIED_SINCE';
        } else if (is304) {
            if (md_conditions.if_modified_since) err.rpc_code = 'IF_MODIFIED_SINCE';
            else if (md_conditions.if_none_match_etag) err.rpc_code = 'IF_NONE_MATCH_ETAG';
        }
    }
}

68-74: Update the STS S3 client factory call to the correct export

The createSTSS3Client function does not exist in cloud_utils.js; the proper export is createSTSS3SDKv3Client. This mismatch will cause a runtime error.

• In src/sdk/namespace_s3.js at line 73, change:

- this.s3 = await cloud_utils.createSTSS3Client(this.s3_params, additionalParams);
+ this.s3 = await cloud_utils.createSTSS3SDKv3Client(this.s3_params, additionalParams);
♻️ Duplicate comments (1)
src/sdk/namespace_s3.js (1)

488-497: Don’t stringify PartNumberMarker; pass number or undefined.

The API expects a number. Stringifying is unnecessary and may cause subtle issues. This also reintroduces a previously fixed issue.

-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,
🧹 Nitpick comments (2)
src/sdk/namespace_s3.js (2)

24-29: Doc types: make bucket required in the constructor JSDoc.

Code enforces a required bucket, but JSDoc marks it optional. Align the type to prevent misuse.

-     *      s3_params: import("@aws-sdk/client-s3").S3ClientConfig & {
+     *      s3_params: import("@aws-sdk/client-s3").S3ClientConfig & {
      *          access_mode?: string,
      *          aws_sts_arn?: string,
      *      },
-     *      bucket?: string,
+     *      bucket: string,
      *      stats: import('./endpoint_stats_collector').EndpointStatsCollector,
      * }} args 

32-43: Access key extraction may be stale with STS/credential providers (affects server-side copy heuristics).

You capture access_key only from s3_params.credentials at construction. If STS is used in _prepare_sts_client (or a provider rotates credentials), this.access_key may not reflect the actual key used by this.s3, leading to false negatives/positives in is_server_side_copy.

Consider one of:

  • Resolve and cache the current access key from the client provider when needed (e.g., a helper get_current_access_keyId() that awaits this.s3.config.credentials()).
  • Or, relax the server-side copy heuristic to attempt CopyObject and fall back on AccessDenied, removing the potentially brittle key comparison.

If you want, I can provide a small helper that memoizes the resolved access key with a short TTL.

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between c9abf9c and efb51fc.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (0 hunks)
  • src/sdk/namespace_s3.js (34 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (4)
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/util/cloud_utils.js
  • src/endpoint/s3/s3_utils.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/sdk/object_sdk.js (1)
src/sdk/namespace_s3.js (2)
  • noobaa_s3_client (14-14)
  • config (8-8)
src/sdk/namespace_s3.js (3)
src/util/cloud_utils.js (6)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • dbg (3-3)
  • _ (11-11)
src/sdk/object_sdk.js (4)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • dbg (9-9)
  • _ (4-4)
src/server/system_services/bucket_server.js (14)
  • noobaa_s3_client (41-41)
  • s3_params (2048-2058)
  • bucket (343-343)
  • bucket (357-357)
  • bucket (376-376)
  • bucket (394-394)
  • bucket (419-419)
  • bucket (431-431)
  • bucket (446-446)
  • bucket (482-482)
  • bucket (491-491)
  • req (1116-1116)
  • req (1149-1149)
  • req (2013-2013)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-package-lock-validation
  • GitHub Check: run-jest-unit-tests
🔇 Additional comments (3)
src/sdk/object_sdk.js (1)

28-28: LGTM: importing noobaa_s3_client for v3 request handler

Import looks correct and aligns with the move to v3 and custom request handlers.

src/sdk/namespace_s3.js (2)

190-221: LGTM: v3-compatible InvalidRange fallback in read_object_md.

Switching to err.name/http 416 detection for InvalidRange is correct for v3 and resolves zero-length object range issues.


808-813: LGTM: union return typing in _get_s3_object_info is v3-aligned.

Accounting for ObjectSize and GetObjectAttributesOutput is good. Size fallback order looks sane.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

♻️ Duplicate comments (1)
src/sdk/namespace_s3.js (1)

490-499: Pass numeric PartNumberMarker; don’t stringify.

The API expects a number. Converting to string is unnecessary. Keep it as a number or undefined.

-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,
🧹 Nitpick comments (6)
src/sdk/namespace_s3.js (5)

24-33: Make bucket required in JSDoc to match runtime check.

Constructor throws when bucket is missing, but the JSDoc marks it optional. Align the types to avoid confusion.

-     *      bucket?: string,
+     *      bucket: string,

249-256: Narrow the request type for read_object_stream; remove unrelated CopyObject type.

This method only performs getObject. The union with CopyObjectRequest (and even HeadObjectRequest) is misleading.

-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & import("@aws-sdk/client-s3").GetObjectRequest | import("@aws-sdk/client-s3").CopyObjectRequest} */
+        /** @type {import("@aws-sdk/client-s3").GetObjectRequest} */
         const request = {
             Bucket: this.bucket,
             Key: params.key,
             Range: params.end ? `bytes=${params.start}-${params.end - 1}` : undefined,
             PartNumber: params.part_number,
         };

367-369: Fix log message typo and improve clarity.

Small nit and readability fix.

-                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and 
-                    key : ${params.key}, with erro : `, err);
+                dbg.error(
+                    `upload_object: failed for bucket ${this.bucket}, key ${params.key}, error:`,
+                    err
+                );

810-816: Type union references look good for v3 shapes; minor doc nit.

Using import("@aws-sdk/client-s3")._Object et al. is accurate for v3. Consider avoiding leading-underscore types in JSDoc if you want to keep API surface stable (non-blocking).


888-912: JSDoc types reference v2 (AWS.S3.*); update to v3 equivalents.

The encryption-related JSDoc still points to v2 types. Replace with v3 S3 shapes (or just document field names) to avoid confusion.

Example:

-     *   ServerSideEncryption: AWS.S3.ServerSideEncryption,
-     *   SSEKMSKeyId: AWS.S3.SSEKMSKeyId,
-     *   SSECustomerKey: AWS.S3.SSECustomerKey,
-     *   SSECustomerAlgorithm: AWS.S3.SSECustomerAlgorithm,
-     *   CopySourceSSECustomerKey: AWS.S3.CopySourceSSECustomerKey,
-     *   CopySourceSSECustomerAlgorithm: AWS.S3.CopySourceSSECustomerAlgorithm,
+     *   ServerSideEncryption: import("@aws-sdk/client-s3").ServerSideEncryption,
+     *   SSEKMSKeyId: string,
+     *   SSECustomerKey: string,
+     *   SSECustomerAlgorithm: string,
+     *   CopySourceSSECustomerKey: string,
+     *   CopySourceSSECustomerAlgorithm: string,
src/sdk/object_sdk.js (1)

472-475: Optional: clarify interplay with NamespaceS3’s v3-only client creation.

NamespaceS3 unconditionally creates a v3 client via get_s3_client_v3_params. If any SigV2-only endpoints remain, they won’t be handled by this namespace regardless of signatureVersion. If that’s intentional per PR scope, ignore; otherwise consider routing SigV2 resources to an alternate namespace/client.

Would you like me to scan the repo for any remaining AWS_V2/SigV2-only configurations and list buckets/resources that could hit this path?

📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between efb51fc and 9c1410f.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (0 hunks)
  • src/sdk/namespace_s3.js (34 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (4)
  • src/endpoint/s3/s3_utils.js
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/util/cloud_utils.js
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/sdk/object_sdk.js (2)
src/sdk/namespace_s3.js (2)
  • noobaa_s3_client (14-14)
  • cloud_utils (10-10)
src/util/cloud_utils.js (1)
  • noobaa_s3_client (16-16)
src/sdk/namespace_s3.js (2)
src/util/cloud_utils.js (6)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • dbg (3-3)
  • _ (11-11)
src/server/system_services/bucket_server.js (14)
  • noobaa_s3_client (41-41)
  • s3_params (2048-2058)
  • bucket (343-343)
  • bucket (357-357)
  • bucket (376-376)
  • bucket (394-394)
  • bucket (419-419)
  • bucket (431-431)
  • bucket (446-446)
  • bucket (482-482)
  • bucket (491-491)
  • req (1116-1116)
  • req (1149-1149)
  • req (2013-2013)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: run-package-lock-validation
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-jest-unit-tests
🔇 Additional comments (5)
src/sdk/namespace_s3.js (3)

190-221: Good v3-compatible InvalidRange fallback and metadata logging.

The 416 detection via err.name and $metadata.httpStatusCode is correct for SDK v3, and the inline GET fallback to HEAD for zero-length objects is appropriate.


259-288: Streaming fix looks good — return a live Readable and propagate errors.

Using the Node Readable (obj_out.Body), piping through a tap for stats, and rethrowing on errors is the right approach. Error bridging from source to returned stream is also a good safeguard.


377-380: ETag extraction is correct; date may be undefined for copy path (acceptable).

Using res.ETag || res.CopyObjectResult?.ETag handles both upload and copy. Returning date only for PUT is fine given the constraints.

src/sdk/object_sdk.js (2)

28-29: Importing noobaa_s3_client here is fine; path is consistent with other modules.

No issues.


466-476: s3_params v3 shape looks correct; signatureVersion reintroduced for compatibility.

  • credentials nested object, region, forcePathStyle, requestHandler, and requestChecksumCalculation are all appropriate for v3.
  • Adding signatureVersion via cloud_utils.get_s3_endpoint_signature_ver(...) restores compatibility with SigV2 endpoints in paths that rely on it.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
src/sdk/namespace_s3.js (1)

496-497: Remove unnecessary .toString() on PartNumberMarker
The params.num_marker?.toString() call is redundant—optional chaining already handles undefined, and other numeric params (e.g. MaxParts) are passed directly. Please update:

• File: src/sdk/namespace_s3.js, line 496

-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 9c1410f and 02ff2f8.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (0 hunks)
  • src/sdk/namespace_s3.js (34 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (4)
  • src/endpoint/s3/s3_utils.js
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/util/cloud_utils.js
🧰 Additional context used
🧬 Code Graph Analysis (2)
src/sdk/namespace_s3.js (3)
src/sdk/object_sdk.js (3)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • _ (4-4)
src/util/cloud_utils.js (5)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • _ (11-11)
src/server/system_services/bucket_server.js (14)
  • noobaa_s3_client (41-41)
  • s3_params (2048-2058)
  • bucket (343-343)
  • bucket (357-357)
  • bucket (376-376)
  • bucket (394-394)
  • bucket (419-419)
  • bucket (431-431)
  • bucket (446-446)
  • bucket (482-482)
  • bucket (491-491)
  • req (1116-1116)
  • req (1149-1149)
  • req (2013-2013)
src/sdk/object_sdk.js (2)
src/sdk/namespace_s3.js (4)
  • noobaa_s3_client (14-14)
  • require (15-15)
  • config (8-8)
  • cloud_utils (10-10)
src/util/cloud_utils.js (1)
  • noobaa_s3_client (16-16)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-jest-unit-tests
  • GitHub Check: run-package-lock-validation
🔇 Additional comments (11)
src/sdk/namespace_s3.js (8)

14-15: LGTM! SDK v3 migration correctly implemented.

The imports are properly structured with the noobaa_s3_client for dynamic request handling and PutObjectCommand for command-scoped middleware usage.


33-45: LGTM! Constructor properly validates and stores bucket parameter.

The bucket validation and explicit bucket storage align well with the v3 migration. The access_key extraction correctly handles the nested credentials structure from v3-style s3_params.


93-93: Minor: Remove redundant promise() calls in list operations.

The list operations have been correctly migrated to v3 style, but ensure all .promise() calls have been removed. The current implementation looks correct.


214-222: LGTM! Improved error handling for InvalidRange with v3 compatibility.

The error handling correctly checks both SDK v3 error patterns (err.name and HTTP status code) and falls back appropriately for zero-length objects.


250-289: LGTM! Stream handling properly implemented for v3.

The implementation correctly:

  • Uses v3 getObject directly without .promise()
  • Returns a live readable stream through the stats tap
  • Properly handles error propagation between source and destination streams
  • Maintains the streaming nature for efficient memory usage

378-380: LGTM! Improved ETag handling for both copy and non-copy operations.

The ETag extraction correctly handles both res.ETag (for PutObject) and res.CopyObjectResult?.ETag (for CopyObject), and the date capture from middleware is properly returned.


827-827: LGTM! Object size calculation enhanced for v3.

The size calculation correctly includes res.ObjectSize in addition to the existing fields, properly handling different response types from v3 APIs.


352-376: No command-scoped middleware leakage

The saveDateMiddleware is added directly to the PutObjectCommand instance, which is created anew for each request. There’s no modification of the shared S3 client’s middleware stack, so the middleware cannot persist across calls. No further action required.

src/sdk/object_sdk.js (3)

28-28: LGTM! Proper import of noobaa_s3_client for v3 support.

The import correctly adds the noobaa_s3_client module needed for dynamic request handler generation in the v3 migration.


461-479: LGTM! S3 parameters correctly migrated to v3 structure.

The s3_params object has been properly updated for SDK v3:

  • Credentials are now nested under credentials object
  • region is explicitly set with fallback to DEFAULT_REGION
  • forcePathStyle replaces s3ForcePathStyle
  • requestHandler uses the noobaa_s3_client for dynamic agent handling
  • requestChecksumCalculation is properly configured
  • signatureVersion is preserved for compatibility

477-478: LGTM! Bucket parameter correctly moved to top-level.

The bucket is now properly passed as a top-level parameter to the NamespaceS3 constructor, aligning with the v3 migration where bucket is no longer nested in s3_params.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/sdk/namespace_s3.js (2)

821-829: Potential crash when res.Metadata is undefined; avoid mutating source.

_.extend(res.Metadata, ...) will throw when Metadata is undefined (common for list/versions/parts outputs) and mutates the response. Use a fresh object.

-        const xattr = _.extend(res.Metadata, {
-            'noobaa-namespace-s3-bucket': this.bucket,
-        });
+        const xattr = _.assign({}, res.Metadata, {
+            'noobaa-namespace-s3-bucket': this.bucket,
+        });

Optionally make LastModified parsing defensive:

-        const last_modified_time = res.LastModified ? res.LastModified.getTime() : Date.now();
+        const lm = res.LastModified;
+        const last_modified_time = lm instanceof Date ? lm.getTime() : (lm ? new Date(lm).getTime() : Date.now());

191-206: Inline GET path must buffer first_range_data when using v3

With v3 the GetObjectOutput.Body is a Node.js Readable stream, so setting request.Range alone won’t populate first_range_data. In your current inline‐GET branch you never consume the stream, and later you do:

first_range_data: Buffer.isBuffer(res.Body) ? res.Body : undefined,

which always yields undefined. Callers in namespace_cache.js and the S3 GET handler rely on first_range_data, so this is now broken.

Mandatory fix in src/sdk/namespace_s3.js (around lines 191–206): collect up to INLINE_MAX_SIZE bytes from the stream before returning. For example:

  if (can_use_get_inline) {
      request.Range = `bytes=0-${config.INLINE_MAX_SIZE - 1}`;
-     // fall through without buffering
+     // v3 returns a Readable; buffer it so first_range_data picks it up
+     const { Body: stream } = await this.client.send(new GetObjectCommand(request));
+     const firstChunk = await stream_utils.collect_stream(stream, config.INLINE_MAX_SIZE);
+     // override Body so Buffer.isBuffer() sees a Buffer
+     res.Body = firstChunk;
  }
  this._set_md_conditions(params, request);

Alternatively, if you only need metadata and no inline bytes, switch to a HeadObjectCommand.

• Location: src/sdk/namespace_s3.js, inline GET branch (≈191–206)
• Affected behavior: first_range_data remains undefined under v3
• Impacted consumers:
– namespace_cache.js (caching when inline data fits)
– src/endpoint/s3/ops/s3_get_object.js (serving small reads)

♻️ Duplicate comments (4)
src/sdk/namespace_s3.js (4)

214-221: V3-compatible InvalidRange handling: LGTM.

Falling back to headObject on err.name === 'InvalidRange' or httpStatusCode === 416 is correct for SDK v3.


270-284: Error propagation on stream: LGTM.

Wiring the source 'error' to the returned stream avoids silent failures. Good fix.


352-366: Command-scoped middleware for Date capture: LGTM.

Moving from client-level to command-level middleware removes race/leak hazards.


491-499: Do not stringify PartNumberMarker; pass number or leave undefined.

SDK expects a number. Stringifying is unnecessary and can produce subtle API errors.

-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,
🧹 Nitpick comments (8)
src/sdk/namespace_s3.js (8)

25-31: Make bucket required in JSDoc to match runtime validation.

Constructor throws when bucket is missing, but JSDoc marks it optional. Align the contract.

-     *      bucket?: string,
+     *      bucket: string,

36-39: Harden access key extraction for server-side copy checks.

In v3, credentials may be provided via a provider function or resolved on the client. Using only s3_params.credentials?.accessKeyId can yield undefined and cause incorrect is_server_side_copy decisions.

Apply this small reorder to derive from client config as a fallback (and tolerate provider functions):

-        this.access_key = s3_params.credentials?.accessKeyId;
         this.endpoint = s3_params.endpoint;
-        this.s3 = noobaa_s3_client.get_s3_client_v3_params(s3_params);
+        this.s3 = noobaa_s3_client.get_s3_client_v3_params(s3_params);
+        {
+            const creds = s3_params.credentials ?? this.s3?.config?.credentials;
+            this.access_key = typeof creds === 'function' ? undefined : creds?.accessKeyId;
+        }

If you intentionally rely only on explicit s3_params.credentials, please confirm so we can drop this suggestion.


191-198: Narrow the request type for clarity.

read_object_md builds either a HeadObjectRequest or GetObjectRequest. Keeping a union in JSDoc is fine, but the current combined type with a single object can confuse tooling.

-            /** @type {import("@aws-sdk/client-s3").HeadObjectRequest | import("@aws-sdk/client-s3").GetObjectRequest} */
+            /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & Partial<import("@aws-sdk/client-s3").GetObjectRequest>} */

This keeps shared fields while acknowledging optional Range.


250-257: Fix JSDoc type: CopyObjectRequest is unrelated here.

This request is only for getObject; including CopyObjectRequest is misleading.

-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & import("@aws-sdk/client-s3").GetObjectRequest | import("@aws-sdk/client-s3").CopyObjectRequest} */
+        /** @type {import("@aws-sdk/client-s3").GetObjectRequest} */

265-267: Remove redundant 3xx guard.

In v3, non-2xx responses throw. The explicit httpStatusCode >= 300 check won’t be reached and adds noise.

-            // In v3, non-2xx typically throws; keep this guard harmless.
-            if (obj_out.$metadata.httpStatusCode >= 300) throw new Error(`S3 getObject failed with status ${obj_out.$metadata.httpStatusCode}`);

367-369: Fix typo and improve error log formatting.

Minor nit: “erro” → “error”, and avoid multiline template literal with trailing spaces.

-                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and 
-                    key : ${params.key}, with erro : `, err);
+                dbg.error(
+                    `upload_object: Object upload failed for bucket ${this.bucket} and key ${params.key}:`,
+                    err
+                );

862-876: Adapt error translation to v3 error shapes.

In v3, err.code is often unset. Prefer err.name and $metadata.httpStatusCode.

-    _translate_error_code(params, err) {
-        if (err.code === 'NoSuchKey') err.rpc_code = 'NO_SUCH_OBJECT';
-        else if (err.code === 'NotFound') err.rpc_code = 'NO_SUCH_OBJECT';
-        else if (err.code === 'InvalidRange') err.rpc_code = 'INVALID_RANGE';
+    _translate_error_code(params, err) {
+        const name = err?.code || err?.name;
+        const http = err?.$metadata?.httpStatusCode;
+        if (name === 'NoSuchKey' || name === 'NotFound' || http === 404) err.rpc_code = 'NO_SUCH_OBJECT';
+        else if (name === 'InvalidRange' || http === 416) err.rpc_code = 'INVALID_RANGE';
         else if (params.md_conditions) {
             const md_conditions = params.md_conditions;
-            if (err.code === 'PreconditionFailed') {
+            if (name === 'PreconditionFailed' || http === 412) {
                 if (md_conditions.if_match_etag) err.rpc_code = 'IF_MATCH_ETAG';
                 else if (md_conditions.if_unmodified_since) err.rpc_code = 'IF_UNMODIFIED_SINCE';
-            } else if (err.code === 'NotModified') {
+            } else if (name === 'NotModified' || http === 304) {
                 if (md_conditions.if_modified_since) err.rpc_code = 'IF_MODIFIED_SINCE';
                 else if (md_conditions.if_none_match_etag) err.rpc_code = 'IF_NONE_MATCH_ETAG';
             }
         }
     }

879-912: Update JSDoc encryption types to v3 or simplify to primitives.

Current annotations reference v2 AWS.S3.* types. In v3, these are simple string fields. Simplify to avoid confusion.

-    /** 
-     * @param {Partial<{
-     *   ServerSideEncryption: AWS.S3.ServerSideEncryption,
-     *   SSEKMSKeyId: AWS.S3.SSEKMSKeyId,
-     *   SSECustomerKey: AWS.S3.SSECustomerKey,
-     *   SSECustomerAlgorithm: AWS.S3.SSECustomerAlgorithm,
-     *   CopySourceSSECustomerKey: AWS.S3.CopySourceSSECustomerKey,
-     *   CopySourceSSECustomerAlgorithm: AWS.S3.CopySourceSSECustomerAlgorithm,
-     *  }>} request
-     */
+    /**
+     * @param {Partial<{
+     *   ServerSideEncryption: import('@aws-sdk/client-s3').ServerSideEncryption | string,
+     *   SSEKMSKeyId: string,
+     *   SSECustomerKey: string,
+     *   SSECustomerAlgorithm: string,
+     *   CopySourceSSECustomerKey: string,
+     *   CopySourceSSECustomerAlgorithm: string,
+     * }>} request
+     */
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 02ff2f8 and 34d1019.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (0 hunks)
  • src/sdk/namespace_s3.js (34 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (4)
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/util/cloud_utils.js
  • src/endpoint/s3/s3_utils.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/sdk/object_sdk.js
🧰 Additional context used
🧬 Code Graph Analysis (1)
src/sdk/namespace_s3.js (3)
src/util/cloud_utils.js (6)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • dbg (3-3)
  • _ (11-11)
src/server/system_services/bucket_server.js (14)
  • noobaa_s3_client (41-41)
  • s3_params (2048-2058)
  • bucket (343-343)
  • bucket (357-357)
  • bucket (376-376)
  • bucket (394-394)
  • bucket (419-419)
  • bucket (431-431)
  • bucket (446-446)
  • bucket (482-482)
  • bucket (491-491)
  • req (1116-1116)
  • req (1149-1149)
  • req (2013-2013)
src/sdk/object_sdk.js (4)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • dbg (9-9)
  • _ (4-4)

};
this._set_md_conditions(params, request);
this._assign_encryption_to_request(params, request);
try {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@naveenpaul1 did you validate this flow? Does it work as expected and update the counter?

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (2)
src/sdk/namespace_s3.js (2)

56-59: Make server-side copy decision robust: require real, equal access keys.

Avoid permitting server-side copy when both access keys are undefined.

-        return other instanceof NamespaceS3 &&
-            this.endpoint === other.endpoint &&
-            this.access_key === other.access_key;
+        return other instanceof NamespaceS3 &&
+            this.endpoint === other.endpoint &&
+            !!this.access_key &&
+            this.access_key === other.access_key;

862-877: v3 error mapping misses err.name/http status; improve translate_error_code.

In v3, err.code is often unset; names and $metadata.httpStatusCode are reliable. Map 404/416 and precondition errors by name/status.

-        const err_code = err.code || err.Code;
-        if (err_code === 'NoSuchKey') err.rpc_code = 'NO_SUCH_OBJECT';
-        else if (err_code === 'NotFound') err.rpc_code = 'NO_SUCH_OBJECT';
-        else if (err_code === 'InvalidRange') err.rpc_code = 'INVALID_RANGE';
+        const http = err?.$metadata?.httpStatusCode;
+        const err_code = err.code || err.Code || err.name;
+        if (err_code === 'NoSuchKey' || err_code === 'NotFound' || http === 404) err.rpc_code = 'NO_SUCH_OBJECT';
+        else if (err_code === 'InvalidRange' || http === 416) err.rpc_code = 'INVALID_RANGE';
         else if (params.md_conditions) {
             const md_conditions = params.md_conditions;
-            if (err_code === 'PreconditionFailed') {
+            if (err_code === 'PreconditionFailed' || http === 412) {
                 if (md_conditions.if_match_etag) err.rpc_code = 'IF_MATCH_ETAG';
                 else if (md_conditions.if_unmodified_since) err.rpc_code = 'IF_UNMODIFIED_SINCE';
-            } else if (err_code === 'NotModified') {
+            } else if (err_code === 'NotModified' || http === 304) {
                 if (md_conditions.if_modified_since) err.rpc_code = 'IF_MODIFIED_SINCE';
                 else if (md_conditions.if_none_match_etag) err.rpc_code = 'IF_NONE_MATCH_ETAG';
             }
         }
♻️ Duplicate comments (2)
src/sdk/namespace_s3.js (2)

496-499: Don't stringify PartNumberMarker; pass number or leave undefined.

This was previously flagged; still using toString(). v3 expects a number; optional chaining avoids a throw but the type is still off.

-            PartNumberMarker: params.num_marker?.toString(),
+            PartNumberMarker: params.num_marker,

36-36: Access key extraction is too narrow; can be undefined in v3 (providers). Gate server-side copy accordingly.

In v3, credentials may come from a provider, so s3_params.credentials?.accessKeyId can be undefined. If both sides are undefined, current equality check erroneously allows server-side copy. Store access_key only when known and require truthy keys in the comparison.

-        this.access_key = s3_params.credentials?.accessKeyId;
+        // Only capture a static access key if present; avoid treating "both undefined" as equal later
+        this.access_key = (s3_params.credentials && 'accessKeyId' in s3_params.credentials)
+            ? s3_params.credentials.accessKeyId
+            : undefined;
🧹 Nitpick comments (10)
src/sdk/namespace_s3.js (10)

29-31: JSDoc mismatch: bucket is enforced by ctor but marked optional.

Constructor throws when bucket is missing, so the JSDoc should declare bucket as required for accuracy and tooling.

- *      bucket?: string,
+ *      bucket: string,

87-94: Consider listObjectsV2 to align with AWS recommendations.

Not blocking, but v2 API (ListObjectsV2) is the modern, paginated call (ContinuationToken instead of Marker) and generally preferred.


250-257: Narrow request typing; CopyObjectRequest is unrelated here.

This path only calls getObject. Tighten the JSDoc to reduce confusion for maintainers and editors.

-        /** @type {import("@aws-sdk/client-s3").HeadObjectRequest & import("@aws-sdk/client-s3").GetObjectRequest | import("@aws-sdk/client-s3").CopyObjectRequest} */
+        /** @type {import("@aws-sdk/client-s3").GetObjectRequest} */

367-369: Typo in error log message.

Minor, but noisy in logs.

-                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and 
-                    key : ${params.key}, with erro : `, err);
+                dbg.error(`upload_object: Object upload failed for bucket ${this.bucket} and key: ${params.key}, with error:`, err);

823-826: Avoid mutating res.Metadata when building xattr.

Safer to copy into a new object; prevents accidental mutation and handles undefined metadata.

-        const xattr = _.extend(res.Metadata, {
-            'noobaa-namespace-s3-bucket': this.bucket,
-        });
+        const xattr = Object.assign(
+            {},
+            res.Metadata || {},
+            { 'noobaa-namespace-s3-bucket': this.bucket }
+        );

234-241: Include err.name/http status in issues report for v3.

err.code may be empty in v3. Include name or http status to make the report useful.

-                object_sdk.rpc_client.pool.update_issues_report({
-                    namespace_resource_id: this.namespace_resource_id,
-                    error_code: String(err.code),
-                    time: Date.now(),
-                });
+                object_sdk.rpc_client.pool.update_issues_report({
+                    namespace_resource_id: this.namespace_resource_id,
+                    error_code: String(err.code || err.name || err?.$metadata?.httpStatusCode),
+                    time: Date.now(),
+                });

370-374: Same as above: enrich error_code for issues report.

-                object_sdk.rpc_client.pool.update_issues_report({
-                    namespace_resource_id: this.namespace_resource_id,
-                    error_code: String(err.code),
-                    time: Date.now(),
-                });
+                object_sdk.rpc_client.pool.update_issues_report({
+                    namespace_resource_id: this.namespace_resource_id,
+                    error_code: String(err.code || err.name || err?.$metadata?.httpStatusCode),
+                    time: Date.now(),
+                });

474-480: Same as above: enrich error_code for issues report.

-                object_sdk.rpc_client.pool.update_issues_report({
-                    namespace_resource_id: this.namespace_resource_id,
-                    error_code: String(err.code),
-                    time: Date.now(),
-                });
+                object_sdk.rpc_client.pool.update_issues_report({
+                    namespace_resource_id: this.namespace_resource_id,
+                    error_code: String(err.code || err.name || err?.$metadata?.httpStatusCode),
+                    time: Date.now(),
+                });

792-799: Same as above: enrich error_code for issues report.

-                object_sdk.rpc_client.pool.update_issues_report({
-                    namespace_resource_id: this.namespace_resource_id,
-                    error_code: String(err.code),
-                    time: Date.now(),
-                });
+                object_sdk.rpc_client.pool.update_issues_report({
+                    namespace_resource_id: this.namespace_resource_id,
+                    error_code: String(err.code || err.name || err?.$metadata?.httpStatusCode),
+                    time: Date.now(),
+                });

51-55: Comment typo.

“aboid” -> “avoid”.

-    // to aboid ACCESS_DENIED errors. a more complete solution is to always perform the server side copy
+    // to avoid ACCESS_DENIED errors. a more complete solution is to always perform the server side copy
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL
Plan: Pro

💡 Knowledge Base configuration:

  • MCP integration is disabled by default for public repositories
  • Jira integration is disabled by default for public repositories
  • Linear integration is disabled by default for public repositories

You can enable these sources in your CodeRabbit configuration.

📥 Commits

Reviewing files that changed from the base of the PR and between 34d1019 and f40f447.

📒 Files selected for processing (6)
  • src/endpoint/s3/s3_utils.js (0 hunks)
  • src/sdk/namespace_s3.js (35 hunks)
  • src/sdk/object_sdk.js (2 hunks)
  • src/test/integration_tests/api/s3/test_s3_ops.js (0 hunks)
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js (0 hunks)
  • src/util/cloud_utils.js (0 hunks)
💤 Files with no reviewable changes (4)
  • src/endpoint/s3/s3_utils.js
  • src/test/integration_tests/api/s3/test_s3_ops.js
  • src/test/unit_tests/util_functions_tests/test_cloud_utils.js
  • src/util/cloud_utils.js
🚧 Files skipped from review as they are similar to previous changes (1)
  • src/sdk/object_sdk.js
🧰 Additional context used
🧬 Code Graph Analysis (1)
src/sdk/namespace_s3.js (3)
src/util/cloud_utils.js (6)
  • noobaa_s3_client (16-16)
  • require (9-9)
  • require (13-13)
  • require (14-14)
  • dbg (3-3)
  • _ (11-11)
src/sdk/object_sdk.js (4)
  • noobaa_s3_client (28-28)
  • require (27-27)
  • dbg (9-9)
  • _ (4-4)
src/server/system_services/bucket_server.js (14)
  • noobaa_s3_client (41-41)
  • s3_params (2048-2058)
  • bucket (343-343)
  • bucket (357-357)
  • bucket (376-376)
  • bucket (394-394)
  • bucket (419-419)
  • bucket (431-431)
  • bucket (446-446)
  • bucket (482-482)
  • bucket (491-491)
  • req (1116-1116)
  • req (1149-1149)
  • req (2013-2013)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (3)
  • GitHub Check: Build Noobaa Image
  • GitHub Check: run-package-lock-validation
  • GitHub Check: run-jest-unit-tests
🔇 Additional comments (3)
src/sdk/namespace_s3.js (3)

39-42: Bucket validation: LGTM.

Throwing on missing bucket prevents "undefined" string leakage and downstream flakiness.


191-221: InvalidRange handling updated for v3: LGTM.

Using err.name and 416 http status covers v3 error shapes and correctly falls back to headObject for empty objects.

Also applies to: 222-223


261-289: Streaming fix and error bridging: LGTM.

Using the Node Readable directly, returning a live stream, and propagating source errors addresses the v3 streaming pitfalls.

@naveenpaul1 naveenpaul1 requested a review from jackyalbo August 21, 2025 06:56
@naveenpaul1 naveenpaul1 merged commit 7de3270 into noobaa:master Aug 21, 2025
18 of 19 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants